It is commonly required to serve fresh content to web users, for example, in the form of web pages that are dynamically generated by formatting the latest content retrieved from a database. Examples of such content might include stock quotes, financial transaction data, price lists, news headlines, computer code, and still more. For instance, consider the case where a user visits an online bank. The user logs in, and after looking at his overall position, he might choose to look at his last ten transactions. The bank's server must retrieve the relevant content from its databases or other memory and construct a formatted response for the user to view. Since such archived content is typically (partly or wholly) unique to the user, even with modern database technology, a bank with millions of users would typically incur many seconds of delay in executing the database queries. Indeed, it is typical of each such query to take five seconds or more, even when the servers and the network are lightly loaded. Such delays are a result of latencies in the back-end infrastructure of the networked system, and are inherent in any large-scale system. As a result, users see poor performance.
In order to improve performance, caches are often installed in tandem with the content server. A cache is server or other mechanism that stores and provides content for re-use. By combining a server and a cache, static content that does not change with time can be offloaded to the cache, freeing up the content server to focus primarily on dynamic content. However, since dynamic content (such as transactional and personalized information) changes frequently and across users, it cannot readily be cached and re-used.
One known technique for improving performance of dynamic content involves pre-fetching of web content. For example, pre-fetching has been offered in a commercial implementation by Fireclick (see www.fireclick.com). In Fireclick's implementation, as soon as a user visits a web site, the web server downloads to the user's browser a small piece of software that contains the most popular browsing patterns at the web site. The software is typically a Java applet that executes on the browser in a transparent fashion. The applet downloads popular web content to the user's browser, in anticipation of the user's next request. The downloaded content may be that which other users most frequently download from the page which is currently displayed on the user's browser. For example, if the front page of a newspaper is being displayed, the stories behind the top headlines might also be downloaded. However, because such content has no guaranteed relation to the individual user's actual browsing habits, the network resources connecting the user's browser to the server, and the memory at the user's browser, are loaded with content that the user may not request.